[Fix] Honor temperature in SequenceClassification setwise scoring - #41267

Motivation

#40826 added temperature-scaled scoring, and #38965 added setwise scoring (score_extraction_token_id). #41188 then added CausalLM setwise and threaded temperature into _process_multi_item_extraction_results, so CausalLM setwise honors it. SequenceClassification setwise still does not: both its paths normalize rows in _multi_position_score_rows, which has no temperature. A request with apply_softmax=True and temperature != 1 passes validation, but every row is still normalized at temperature 1, so the value is silently ignored.

Example: pooled logits [1.0, 3.0] with temperature=2.0 should score softmax([0.5, 1.5]) = [0.269, 0.731]. Both SequenceClassification setwise paths (batched and --enable-mis) return [0.119, 0.881].

Pointwise scoring and CausalLM setwise already honor temperature and are unchanged.

Modifications

• _multi_position_score_rows takes a keyword-only temperature and applies the same numerically stable softmax as the pointwise branch: float64, subtract each row's max, then divide by temperature. The argument is required so a new call site cannot silently drop it. With apply_softmax=False, raw logits are returned unchanged, as before.
• Its two callers (the batched setwise branch of _process_single_item_scoring_results and the classification branch of _process_multi_item_extraction_results) pass temperature through; both already receive it.
• Tests:
  • New test_setwise_classification_temperature in test/registered/unit/test_token_scoring.py. It drives score_request end to end through the existing ScoringManager fake (only model execution is faked), for both enable_mis=False and enable_mis=True, with two items that have different anchor counts (1 and 2). It checks every row at temperature=2.0 and at temperature=1e-300. The latter guards the float64 computation: in float32, 1e-300 rounds to 0 and every row becomes NaN. The fake now returns one row per pooled position when token_indices_to_pool is set.
  • test_setwise_score_mixin.py: the three direct calls to _multi_position_score_rows pass temperature=1.0.

Rebased onto main after #41188; the source change is now limited to the row helper and its two call sites.

Accuracy Tests

No model forward, kernel, or pooling code changed; only post-processing of the pooled logits. Temperature-1 results are unchanged up to float precision: the softmax now runs in float64, matching the pointwise branch.

Unit tests (CPU, Python 3.11, torch 2.13.0+cpu, WSL2):

python -m pytest test/registered/unit/managers/test_setwise_score_mixin.py test/registered/unit/test_token_scoring.py -q

Not run: an end-to-end /v1/score smoke test against a real SequenceClassification model.

Speed Tests and Profiling

N/A. There is no performance claim: this changes a small softmax on the already-pooled [num_positions, num_labels] result.

Checklist

• Format your code according to the Format code with pre-commit. (isort 7.0.0, ruff 0.15.1 check + format, and codespell 2.4.1 on the changed files, versions from .pre-commit-config.yaml; the full pre-commit suite was not run.)
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations. (N/A: setwise scoring is not documented under docs/ yet.)
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed. (N/A: no model-output or speed impact beyond the fix itself.)
• Follow the SGLang code style guidance.

CI States

Latest PR Test (Base): ❌ Run #38434296038
Latest PR Test (Extra): ❌ Run #38434296016
Latest PR Test (AMD ROCm 10): ❌ Run #38434295942